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REVIEW  OF  HIGH  MEMORY  DEMAND  COURSES  IN  THE  MILITARY  INTELLIGENCE 
OFFICER  BASIC  COURSE  (MIOBC) :  A  CASE  STUDY 


Introduction 

For  the  past  couple  of  years,  the  instructors  of  the  "threat" 
block  have  suggested  to  ARI ,  Fort  Huachuca  Field  Unit  that  they 
provide  some  assistance  with  problems  observed  in  that  block, 
notably,  the  large  number  of  Criterion  Action  Elements  (CAE)  "drops" 
that  students  experience. 

For  some  time  the  Military  Intelligence  Officer  Basic  Course 
(MIOBC)  "threat”  block  has  presented  difficulties  in  terms  of  the 
large  amount  of  material  that  needs  to  be  learned  in  a  relatively 
short  period  of  time.  The  difficulties  are  manifest  in  the 
inordinately  large  number  of  students  who  have  to  retake 
examinations  over  more  than  a  couple  of  CAEs,  the  not  infrequent 
"recycling"  that  occurs,  and  in  the  reported  retention  problems  that 
students  have  when  they  proceed  to  other  blocks  of  the  OBC  course, 
notably.  Intelligence  Preparation  of  the  Battlefield  (IPB) , 
Processing,  and  War  Exercise  (WAREX) .  The  most  salient  feature, 
given  the  short  duration  of  the  course,  would  seem  to  be  the  need 
for  a  more  efficient  procedure  for  the  rapid  acquisition  and  storage 
of  the  substantial  amount  of  material  contained  in  the  course. 

Preliminary  discussions  with  the  instructors  suggested  that 
students  were  just  not  studying  for  the  course,  and  that  some  had  a 
cavalier  attitude  towards  the  course.  There  were  no  hard  data  to 

substantiate  that  claim.  Neither  were  there  systematic  data  on 
other  aspects  of  students'  performance  in  the  block,  nor  in  the 
relative  difficulties  encountered  in  subsequent  blocks.  The 
"threat"  block  is  considered  an  enabling  task,  providing  an 
essential  basic  foundation  for  subsequent  success  in  other  blocks 
of  the  OBC  course,  and  it  was  considered  appropriate  to  conduct 
an  examination  along  several  dimensions.  This  report  details  the 
findings  from  this  examination. 


Method 

Several  aspects  of  student  performance  were  evaluated.  These 
included: 

1.  Statistical  analysis  of  nine  classes  of  OBC  students, 
yielding  an  analysis  of  the  performance  of  316  students. 

2.  Comparison  data  on  a  smaller  sample  (n=106)  of  students  on 
their  performance  in  IPB  and  Processing. 

3.  Questionnaire  data  from  Class  89-11. 
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4.  Observation  of  course  lectures  and  exercises. 

5.  Discussions  with  course  instructors  and  ARI  colleagues. 


Results 


Statistical  Analysis 

For  the  purpose  of  statistical  analyses,  grade  matrix  data  were 
collected  on  nine  classes  of  OBC  students.  Class  size  varied  from 
27  to  44  students,  and  except  for  the  occasional  recycled  student 
and  reservist  who  had  some  prior  knowledge  of  "threat”  related 
material,  all  students  were  naive  about  the  content  of  the  block. 

The  first  analysis  had  as  its  purpose  the  quantification  of  the 
actual  state  of  affairs  vis  a  vis  success  in  the  block.  The  basic 
datum  considered  was  the  initial  CAE  pass  rate  in  the  course 
(initial  rate  was  chosen  as  the  major  indicator  since  ultimate 
passing  of  dropped  CAEs  is  virtually  "guaranteed"  by  the  retesting 
procedures  of  the  block) ,  and  this  was  further  broken  down  into  the 
success  for  each  particular  CAE  (there  are  12  altogether)  in  the 
block.  Figure  1  shows  the  results  of  that  analysis.  There  is  some 
variation  due  to  the  individual  classes.  These  differences  were 
subjected  to  an  analysis  of  variance  in  which  CAEs  were  considered 
"subjects"  (n=12) .  The  results  of  that  analysis  yielded  a 
significant  difference  among  classes,  £=  3.71,  p  <  .01. 

It  was  considered  that  the  variation  in  class  performance  may 
have  been  due  to  class  size  or  the  quality  of  instruction  in  the 
course,  since  instructors  are  rotated  through.  A  correlational 
analysis  between  class  size  and  overall  mean  passing  rate  for  the 
nine  classes,  however,  yielded  r=  -.38,  p  >  .05,  an  insignificant 
relationship.  To  assess  the  possible  contribution  that  instructor 
differences  may  have  had,  the  mean  performance  for  the  combined 
classes  of  each  of  the  instructors  was  obtained.  There  were  only 
two  instructors  for  the  nine  classes.  Instructor  A's  classes  had  a 
mean  passing  rate  of  77.09  (sd  =  5.17),  and  Instructor  B's  classes 
had  a  rate  of  80.26  (sd  =  5.77).  It  is  reasonable  to  conclude  that 
neither  class  size  nor  instructor  differences  account  for  class 
performance  differences.  Further,  to  whatever  these  differences  may 
be  attributed,  the  actual  per  class  pass  rate  on  CAEs  overall  varied 
between  71.52  and  88.27  percent  (mean  =  79.21,  sd  =  5.48).  The 
former  value  is  very  low,  and  the  latter  value  is  not  that  high,  so 
all  classes  may  be  considered  to  have  experienced  difficulty,  and 
the  remaining  analyses  consider  aggregate  data  for  the  nine  classes 
(n=316  students) . 

The  average  performance,  then,  over  all  nine  classes  is  depicted 
in  Figure  2  in  which  the  average  percentage  of  students  passing  each 
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Figure  1.  Percent  of  students  passing  each  CAE  for  each  of  nine 
classes. 
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C  A  E'  5 


Figure  2.  Average  number  of  students  passing  each  CAE. 
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of  the  CAEs  is  given,  along  with  the  standard  deviation  of  those 
means.  The  major  conclusion  to  be  drawn  from  these  data  is  that 
some  CAEs  are  consistently  passed  at  a  higher  level  than  others.  At 
the  extremes,  CAEs  7  and  8  (representing  "offense")  have  the  highest 
overall  pass  rate,  with  the  least  accompanying  variability,  whereas 
CAE  10  ("airborne”)  has  the  lowest  overall  pass  rate,  and  a  moderate 
level  of  between  classes  variability. 

To  get  a  different  look  at  class  performance,  an  analysis  was 
performed  on  the  number  of  students  that  proceeded  through  the  block 
having  dropped  "n"  number  of  CAEs.  Figure  3  presents  these  data  in 
terms  of  both  absolute  number  of  CAEs  dropped,  and  in  percent. 
Inspection  of  Figure  3  shows  that  a  surprising  number  of  students  in 
the  overall  sample  (at  least  to  one  veteran  instructor)  made  it 
through  the  block  without  dropping  any  CAEs  (95  students,  or  roughly 
30%) ,  although  success  drops  off  steadily  thereafter. 

It  was  then  of  interest  to  determine  how  badly  missed  dropped 
CAEs  actually  were.  Figure  4  presents  these  data.  The  data  in 
Figure  4  are  depicted  in  terms  of  both  the  mean  actual  number  of 
criterion  points  by  which  a  given  CAE  was  dropped,  and  more 
informatively,  because  the  criterion  point  values  vary  among  the 
CAEs,  the  mean  percentage  of  the  criterion  score  by  which  a  CAE  was 
dropped.  For  the  latter,  dropped  CAEs  missed  the  criterion  score  by 
an  average  of  about  24%  of  the  criterion  score,  or  about  18%  of  the 
maximum  score  (of  necessity  this  value  is  lower  since  maximum  point 
value  is  higher  than  criterion  point  value).  Thus,  while  the 
average  number  of  points  by  which  a  CAE  was  dropped  remained 
relatively  constant  at  about  two  points,  this  value  takes  on  a 
greater  weight  for  CAEs  having  lower  point  values. 

In  fact,  since  the  criterion  point  values  per  CAE  vary  between  5 
and  15  (maximum  point  values  between  7  and  21)  an  analysis  was 
performed  to  determine  if  there  was  a  significant  correlation 
between  the  number  of  test  items  in  the  CAEs  and  the  percentage  of 
students  passing  each  CAE.  The  value  of  that  correlation  was  r=.21, 
E  >  .50,  an  insignificant  relationship. 

The  mean  criterion  passing  percentage  required  for  each  CAE 
varied  between  71  and  78  percent  (mean  =  74.58,  sd  =  2.94),  in  each 
instance  slightly  higher  than  the  doctrinal  level  of  70%.  Vhere  was 
not  a  significant  correlation  between  the  mean  passing  level  of  CAE 
and  its  associated  criterion  passing  percentage,  r=  -.48,  p  >  .10, 
althouqh  the  direction,  at  least,  of  this  relationship  is  consistent 
with  tne  fact  that  higher  criterion  percentages  may  be  related  to 
lower  passing  rates  (not  a  surprising  outcome) . 

There  are  two  examinations  in  this  block.  The  first  covers  five 
CAEs  and  the  second  covers  seven  CAEs.  Does  performance  differ  over 
the  two  examinations?  To  answer  this  question,  the  percentage  of 
total  course  CAEs  that  were  dropped  was  considered  vis  a  vis  the 
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Figure  4.  Criterion  points  missed  related  to  dropped  CAEs. 
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expected  number  of  CAEs  to  be  dropped  per  examination.  It  was 
established  that  for  an  unbiased  estimate  5/12  (42%)  of  CAEs  should 
have  been  dropped  on  the  first  examination,  and  7/12  (58%)  should 
have  been  dropped  on  the  second  examination.  The  actual  obtained 
values  were  very  close,  39%  and  61%,  respectively.  So  class 
performance  remained  "even"  over  the  two  examinations  (see  Figure 
5)  • 

Comparison  Data 

Since  anecdotal  information  about  students'  poor  performance  in 
the  "threat"  block  was  the  impetus  for  this  study,  a  comparison  was 
made  between  "threat"  performance,  and  that  for  IPB  and  Processing. 
(IPB  and  Processing  are  the  courses  requiring  cognitive  utilization 
of  threat,  as  well  as  other  material) .  For  this  comparison,  data 
from  three  classes  was  available  (total  n=106  students) .  Figure  6 
presents  the  mean  (and  sd)  percentage  of  students  passing  each  CAE 
for  both  IPB  and  Processing,  and  Figure  7  shows  mean  percent  passing 
each  CAE  for  all  three  blocks.  Performance  in  IPB  seems  to  be 
higher  than  performance  in  Processing  (the  reasons  need  to  be  left 
to  another  analysis) ,  and  the  anecdotal  information  was  correct,  at 
least  in  part.  Performance  in  "threat"  is,  overall,  inferior  to 
performance  in  IPB  and  Processing  for  these  three  classes. 

Questionnaire  Data 

Class  89-11  was  surveyed  with  a  questionnaire  designed  to 
determine  their  feelings  about  various  aspects  of  the  "threat"  block 
as  now  in  place,  and  about  some  possible  modifications  that  might  be 
made  (these  changes  will  be  addressed  in  the  Recommendations  Section 
of  this  report) .  Out  of  the  approximately  30  students  in  that 
class,  12  responded  to  the  questionnaire,  and  only  one  student 
volunteered  for  a  personal  interview  (subsequently  cancelled) . 

These  data  must  therefore  be  considered  only  suggestive  at  best. 

Most  of  the  questions  asked  for  students  to  respond  with  the 
following  seven  point  scale: 

Not  at  all  Very  Much 

1  2  3  4  5  6  7 

The  results  are  summarized  as  follows  with  numbers  in 
parentheses  being  average  response  values: 

1.  Students  found  their  own  class  notes  (6.25)  and 

lecture  outlines  (6.58)  to  be  more  valuable  than  the  course  manuals 
(3.33)  in  preparing  for  the  examinations. 

2.  Those  who  participated  in  study  groups  (8  out  of  12)  found 
them  to  be  moderately  valuable  (5.00). 
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Figure  5.  Expected  vs  observed  CAEs  dropped  by  exams. 
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3.  Students  would  have  preferred  an  examination  over  a  CAE  as 
soon  as  they  had  mastered  the  material  (5.33). 

4.  At  a  moderate  level,  they  would  have  preferred  personal 
evaluations  of  subject  matter  mastery  (4.83). 

5.  Realistic  models  of  Soviet  equipment  and  the  opportunity  to 
manipulate  them  into  various  configurations  and  battlefield 
scenarios  would  have  been  a  helpful  exercise  for  organizing  and 
conceptualizing  the  material  they  had  to  learn  (5.82). 

6.  Of  those  that  have  used  computerized  instructorial  modules, 
students  thought  they  would  be  helpful  (5.33). 

For  the  record,  two  of  the  students  thought  that  nothing  would  have 
helped,  giving  "l's"  for  most  of  their  responses. 

Observation  of  Course  Lectures  and  Exercises 

The  investigator  had  the  opportunity  to  observe  several  lectures 
over  the  course  material  and  to  observe  the  final  WAREX  exercises. 
Lectures  are  preceded  by  the  handing  out  of  lecture  outlines,  some 
of  which  are  in  some  detail.  The  lectures  that  were  observed  were 
accompanied  with  graphic  representations,  and  the  atmosphere  in  the 
lectures  was  open  and  interactive.  There  was  ample  opportunity  for 
student  questions,  and  student  attention  was  at  a  high  level  since 
some  were  called  upon,  on  a  random  basis,  to  answer  questions. 

The  material  as  presented  in  the  instructional  manuals  is 
essentially  verbal  in  nature.  The  presentation  of  the  organization 
and  equipment  matrices  is,  of  necessity,  complex.  The  pictorial 
presentations  are  so  poorly  reproduced  as  to  yield  little  or  no 
visual  memory  for  what  they  represent.  Thus,  virtually  the  whole  of 
what  the  students  have  encoded  is  verbal  in  nature.  They  have  only 
a  limited  concept  of  what  the  symbology  represents,  or  what  the 
equipment  looks  like.  In  essence,  they  have  memorized  by  rote  the 
facts  of  the  material,  but  still  do  not  have  a  personal,  multi¬ 
modal  organizational  or  representational  system  for  that 
information. 

The  WAREX  exercise  was  particularly  illuminating  since  it 
represents  what  the  "final  product"  of  the  OBC  is  supposed  to  be,  it 
was  observed  that  the  students  had  some  difficulty  responding  to 
questions  concerning  Soviet  threat  matters.  It  was  also  apparent 
that  the  colonel  taking  the  briefing  placed  major  importance  on 
intelligence  officers  being  able  to  have  such  information  readily 
available. 

From  the  beginning,  this  observer  was  impressed  by  the  lack  of 
any  sophisticated,  contemporary  instructional  technology  being  used 
in  the  "threat"  block.  What  instructional  materials  there  are 
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consist  of  mimeographed  handouts  prepared  by  the  instructors.  The 
instructional  delivery  system  is  generations  behind  the  material  the 
students  are  supposed  to  be  learning  about.  On  the  other  hand, 
there  have  been  a  couple  of  instruments  that  by  one  means  or  another 
have  been  developed  for  instructional  purposes  that  parallel  the 
objectives  of  the  "threat"  block. 

Several  years  ago,  a  series  of  computer  programs  called  "Think 
Red"  was  developed  that  was  to  represent  in  graphic  form  Soviet 
doctrine.  As  far  as  could  be  ascertained  that  material  was  never 
fully  introduced  into  the  "threat"  course,  nor  was  there  ever  an 
evaluation  of  the  effectiveness  of  it  as  an  instructional  aid.  The 
ARI-Ft.  Huachuca  Field  Unit  did  present  an  evaluation  of  "Think  Red" 
in  terms  of  its  compatibility  with  contemporary  cognitive  theory 
(Hall  &  Knapp,  1983) ,  however,  that  program  is  obsolete.  Also, 
there  is  a  current,  developing  Apple  Macintosh  program  called  "Red 
Star"  (Science  Applications  International  Corporation  (SAIC) )  that 
attempts  to  present  Soviet  doctrine,  but  it  is  uncertain  as  to  what 
the  disposition  of  that  program  will  be.  (Also,  it's  on  a  non¬ 
compatible  software  format) . 

Discussions  with  Course  Instructors  and  ARI  Colleagues 

The  purpose  of  these  discussions  was  to  get  some  idea  of  how  the 
course  has  developed  and  what  attempts  have  been  made  in  that 
history  to  improve  the  final  product. 

Several  points  are  worth  noting: 

1.  The  course  instructors  are  functionally  the  primary 
determiners  of  what  the  specific  content  of  the  course  will  be,  how 
it  will  be  taught,  and  what  resource  material  will  be  available  to 
the  students.  Although  there  is  a  course  manager,  the  function  of 
that  office  seems  to  be  mostly  administrative  in  nature,  in  order  to 
perform  quality  control  functions. 

2.  The  instructors  are  well  motivated  and  well  meaning, 
although  having  to  frequently  rotate  through  new  classes  would  seem 
to  leave  little  time  for  the  development  of  personal  concerns  for  an 
individual  student.  On  the  other  hand,  it  must  be  noted  that  all 
instructors  note  that  they  are  paid  for  "24  hours  work,"  and  are 
available  to  the  students  when  they  need  the  help.  Unfortunately, 
students  are  probably  reluctant  to  bother  the  instructors  on 
evenings  and  on  the  weekend — times  when  the  students  are  presumably 
doing  their  most  intensive  studying.  Not  having  course  resources 
available  during  these  times  is  particularly  disadvantageous 
considering  the  short  duration  of  the  block. 

3.  The  instructors  have  made  several  attempts  on  their  own  to 
improve  the  quality  of  instruction  and  to  promote  better  acquisition 
of  the  course  material.  They  have  prepared  detailed  lecture 
outlines,  developed  "pop  quizzes,"  and  have  tried  to  see  that  all 
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material  on  the  examinations  has  been  covered  explicitly  during  the 
lectures.  However,  some  coverage  has  to  be  left  out  by  virtue  of 
the  amount  of  time  available  for  lectures.  Anecdotally,  it  was 
observed  from  the  WAREX  briefing  that  when  the  range  of  certain 
weapons  was  requested,  an  observing  "threat"  instructor  said  to 
students  standing  by,  "I  told  you  that  you  would  have  to  know  the 
range  of  that  eventually."  Obviously,  then,  in  picking  and  choosing 
among  essential  information  during  "threat"  instruction,  some 
material  had  to  be  left  out.  Indeed,  that  information  is  in  the 
manual,  but  according  to  questionnaire  data,  students  tend  to  spend 
little  time  with  the  manual  per  se. 

4.  The  instructors  have  been  successful  in  securing  a  recent 
increase  in  the  number  of  hours  devoted  to  "threat"  instruction, 
although  they  cannot  point  to  that  having  a  noticeable  increase  in 
student  performance. 

5.  The  current  instructors  have  been  at  their  assignments  for 
at  least  several  years.  There  is  no  scheduled  activity  by  which 
their  own  knowledge  of  "threat"  related  information  is  up-dated. 

They  attempt  to  keep  abreast  by  informal  contacts  with  field  units, 
and  reading  documents  which  they  are  able  to  obtain  on  their  own. 

The  current  frequency  of  turnover  of  instructors  is  not  sufficient 
to  allow  for  "new  blood"  in  the  system.  They  would  welcome  the 
opportunity  to  have  more  contact  with  the  greater  active  MI 
community. 

6.  This  OBC  course,  as  currently  structured  and  administered 
through  channels,  not  only  allows  for  automatic  re-examinations,  but 
virtually  guarantees  that  students  will  eventually  pass  (nearly  all, 
but  there  are  some  exceptions) .  It  would  not  be  difficult  to 
document  instances  of  students  taking  examinations  over  particular 
CAEs  up  to  six  or  more  times.  In  fact,  one  former  instructor 
considered  that  some  students  may  view  the  initial  examination  as 
just  practice,  knowing  full  well  that  they  will  eventually  pass. 

This  is  particularly  troublesome  since  CAEs  dropped  on  the  second 
and  last  exam,  will  have  to  be  made  up  while  the  students  have 
proceeded  to  the  next  part  of  the  course.  Thus,  the  student  is  in 
the  position  of  having  to  double  up  on  studies,  and  so  optimizing 
learning  of  both  subjects  is  in  jeopardy. 


Recommendations 

The  strategy  of  this  course  analysis  has  been  multidimensional, 
and  so  the  recommendations  that  follow  will  address  several 
different  aspects  of  the  course.  These  recommendations  make  the 
assumption  that  course  content  and  duration  are  relatively  fixed. 
The  recommendations  are  not  in  order  of  priority. 
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The  instructors  need  to  have  the  opportunity  to  be 
systematically  kept  abreast  of  developments  in  the  MI  community. 
There  are  a  couple  of  ways  to  accomplish  this.  It  would  help  to 
keep  in  mind  that  "threat"  is  viewed  as  an  enabling  task,  and  as  a 
critical  task,  and  so  it  does  not  have  a  front-end  analysis  of 
course  content.  Materials  for  the  course  are  developed  using 
historical  precedence,  personal  concern  for  updating  of  content,  and 
well-intentioned  consensus. 

It  would  be  useful  to  conduct  a  written  survey  of  MI  Captains 
entering  Ft.  Leavenworth's  Combined  Arms  Services  Staff  School 
(CAS3) .  These  officers  will  have  had  more  than  one  tour,  and  will 
be  able  to  offer  valuable  guidelines  to  be  used  in  the  development 
of  essential  course  content,  as  well  as  providing  valuable  field 
information  for  the  instructors.  Another  way  to  accomplish  this 
objective  would  be  for  the  current  instructors  to  be  able  to  tour, 
inspect,  and  observe  selected  active  MI  units,  or  attend  threat 
training  at  Ft.  Leavenworth  and  the  DIA. 

One  of  the  most  useful  handouts  that  has  been  prepared  for  the 
students  is  a  single  page  summary  of  the  Soviet  military 
organization,  and  the  type  of  equipment  to  be  found  at  each  level. 
This  handout  should  be  expanded  into  a  nearly  complete  description 
of  the  content  of  the  "threat"  block.  All  college  students  are 
familiar  with  the  laminated  sheets  that  can  be  purchased  which  show, 
for  example,  everything  you  might  need  to  know  about  calculus. 
Carefully  prepared  (color  coded,  etc.)  such  a  tool  would  be  a 
valuable  memory  aid  and  reference  instrument  for  continued  use 
(until  upgrading  was  required) .  One  side  could  consist  of  weapons 
information,  and  the  other  would  consist  of  organization  and 
offensive  and  defensive  doctrine.  Upgrading  from  time  to  time  would 
pose  little  problem  for  the  word-processed  package.  Having  had  such 
a  tool  would  have  facilitated  the  mad  scramble  through  the  manuals 
that  occurred  during  a  WAREX  exercise  when  the  colonel  asked  for  the 
range  information.  This  instrument  would  also  serve  to  organize  the 
most  essential  information  for  the  student.  It  is  very  common  for 
individual  students  to  prepare  similarly  conceived  instruments  for 
their  own  study,  but  one  carefully  thought  out  by  SMEs  would  likely 
be  superior  and  more  useful.  Prototype  products  exist,  such  as  the 
one  called  S-2  Miltra  Field  Aid,  but  it  costs  $40.00  per  copy. 

The  students  do  not  come  to  "know"  what  they  are  learning  about. 
The  simple  verbal  descriptions  do  not  yield  a  satisfactory  memory 
representation.  This  observation  is  not  limited  to  the  "threat" 
block  of  instruction.  A  former  instructor  in  IPB  and  Processing  was 
looking  into  the  possibility  of  having  simple  plywood  mock-ups  of 
Soviet  equipment,  so  his  students  would  have  a  better  idea  of  what 
that  equipment  was  like  when  actually  seen  in  the  field.  The 
"threat"  students  also  need  to  have  a  multidimensional 
representation  of  this  equipment,  and  how  it  might  appear  in  a 
battlefield  array.  Slides  of  individual  pieces  of  Soviet  equipment 
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under  repair,  as  valuable  as  they  may  be  for  structural  analysis, 
etc. ,  do  not  convey  the  "feel"  of  the  battlefield  disposition  of 
that  equipment.  It  is  noteworthy  that  the  German  Air  Defense  School 
(Ft.  Bliss,  TX)  also  created  a  field  mock-up  of  Air  Defense  #-D 
Battlefield  Systems  using  models,  wood,  plastic,  etc.  It  was  rather 
costly,  but  got  extensive  use  and  was  considered  "invaluable"  by  the 
local  instructional  staff. 

The  accuracy  of  the  visual  memory  system  can  easily  be 
documented.  For  example,  Standing,  Conezio,  and  Haber  (1970)  have 
shown  that  subjects  can  remember  over  90%  of  complex  visual  scenes, 
even  though  the  number  to  which  they  were  originally  exposed  was 
over  2500.  Further,  the  pictorial  representations  shown  in  the 
"threat"  block  are  of  isolated  pieces  of  equipment,  never  of  the 
complete  battlefield  disposition  of  potential  enemy  forces.  In  this 
regard,  it  should  be  noted  that  interactive  visual  images  are 
particularly  well  remembered  (Bower,  1970;  Wollen,  Weber,  &  Lowry, 
1972) .  Although  there  has  been  some  debate  over  exactly  what  the 
basis  of  these  effects  might  be  (Pylyshyn,  1973,  for  example),  there 
can  be  no  doubt  about  the  efficacy  of  the  visual  experience.  Visual 
experience  is  the  source  of  visual  memory  events.  While  it  may  not 
be  possible  to  obtain  actual  photographs  of  entire  arrays,  and  it  is 
certainly  unlikely  that  firsthand  visual  images  of  actual 
battlefield  dispositions  will  be  made  available  to  "threat" 
students,  the  following  recommendation  is  offered  as  an  alternative 
to  provide  the  visual  contextual  reference  that  is  now  missing  in 
the  "threat"  instruction; 

It  would  be  useful  for  "threat"  students  to  have  a 
laboratory  component  in  their  instructional  block. 

This  laboratory  would  provide  not  only  a  visual 
encoding  of  some  course  information,  but  it  would 
also  provide  an  interactive/contextual  frame  of 
reference  that  is  now  missing.  In  essence  it  is  as 
if  they  were  now  being  told  to  play  chess  after  only 
having  read  about  the  many  moves  possible.  The 
laboratory  exercises  would  consist  of  the  active 
manipulation  of  realistic  models  of  equipment, 
assigned  to  different  Soviet  organizational 
levels,  for  different  terrains,  in  different  defensive 
and  offensive  scenarios,  etc.  The  students  surveyed 
thought  this  would  be  quite  useful.  Having  these 
"table  top"  laboratory  exercises  will  help  to 
establish  the  visual  memory  link  between  the 
verbal  descriptions  and  physical  reality  that  is 
not  now  being  made. 

The  final  recommendation  is  the  most  substantial,  and  will  be  dealt 
with  at  some  length. 
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Given  the  short  duration  of  this  block  of  instruction  and  the 
large  amount  of  information  that  must  be  learned,  an  upgrade  to  the 
instructional  technology  is  appropriate  and  timely.  The 
recommendations  above  will  improve  the  students'  mental 
representation  and  organization  of  the  information,  but  much  more 
can  be  done.  A  computerized  tutorial  package  is  recommended.  Many 
universities  have  adopted  variations  of  this  approach  for  a  variety 
of  courses  and  they  have  met  with  some  success.  Improvement  in  long 
term  retention  in  some  studies  (Kulik,  Bangert,  &  Williams,  1983; 
Niemiec  &  Walberg,  1987)  has  been  documented  to  be  about  16%,  and 
some  have  the  added  advantage  of  achieving  accelerating  learning 
rate  through  a  self-paced,  unit  mastery  learning  approach.  Further, 
in  general,  student  and  instructor  attitudes  towards  computer 
assisted  instruction  are  also  positive  (Kulik,  et  al.,  1983; 

Halcomb,  Chatfield,  Stewart,  Stokes,  Cruse,  &  Weimer,  in  press) . 

All  of  this  would  seem  to  be  important  for  the  "threat"  block  given 
the  extensive  amount  of  material  to  be  covered  and  its  very  short 
duration. 

Computer  assisted  instruction  may  be  implemented  in  a  variety  of 
formats.  There  may  be  a  complete  computer  controlled  application  in 
which  the  computer  not  only  presents  the  information  for  study  and 
tests,  but  in  which  the  computer  is  used  for  tutorials  as  well  as 
providing  a  complete  course  management  system.  Many  permutations 
are  also  possible.  Students  may  study  from  traditional  texts, 
attend  lectures,  and  use  the  computer  only  for  testing  or  tutorials. 
Or,  any  one  of  these  tasks  may  be  relegated  to  the  computer  as  best 
fits  the  needs  of  the  course. 

Any  computer  assistance  application  will  be  useful  to  the  extent 
that  the  developers  have  considered  the  many  factors  that  have  been 
shown  to  determine  the  effectiveness  of  such  systems.  Among  these 
factors  are  (1)  aspects  of  the  program  that  affect  reading 
comprehension  (Anderson,  Anderson,  Dalgaard,  et  al.,  1975;  Schloss, 
Schloss,  &  Cartwright,  1984,  1985),  (2)  the  appropriate  application 
of  testing  in  the  overall  learning  process  (Foos  &  Fisher,  1988) , 

(3)  the  nature  of  the  feedback  during  the  instructional  phase 
(Gilman,  1969;  Anderson  et  al.,  1971;  Cohen,  1985),  and  (4)  the 
presentation  of  the  feedback  (Kulhavy  &  Anderson,  1972) .  There  are 
other  factors,  but  this  list  should  make  the  point  that  the 
development  of  a  successful  computer  assisted  instruction 
application  is  not  simply  a  matter  of  possessing  programming  skills, 
therefore,  it  must  be  purchased  "commercially"  rather  than  produced 
in-house. 

The  Self  Test  and  Review  (STAR)  package,  developed  by  Dr. 

Charles  Halcomb  and  Associates  at  Texas  Tech  University  (Halcomb  et 
al.,  in  press),  has  the  advantage  of  having  been  developed,  and  is 
currently  being  used,  in  both  an  instructional  and  a  research 
setting,  so  analysis  and  upgrading  of  the  system  is  ongoing.  It 
will  be  reviewed  here  as  an  example  of  a  successful  application. 
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STAR  was  designed  as  a  study  tool  to  be  used  with  the  course 
textbook  and  study  guides.  The  basic  goal  of  STAR  is  to  focus  on 
important  concepts  in  the  course  and  to  provide  knowledge  feedback 
to  the  student  so  that  they  have  a  better  idea  of  what  they  do  and 
do  not  know.  (Students  are  relatively  poor  assessors  of  their  own 
comprehension  level) .  Basically,  STAR  offers  practice  quizzes  and 
final  examinations,  a  guided  review  of  chapter  material  and 
concepts,  and  graphic  representations  of  student  quiz  performance. 
Procedural ly,  each  student  has  their  own  copy  of  the  STAR  program 
which  can  run  on  any  IBM  compatible  machine. 

To  document  the  effectiveness  of  the  STAR  program,  a  brief 
review  of  the  latest  evaluation  study  will  be  presented.  This  study 
is  the  doctoral  dissertation  project  of  Ms  Barbara  Stewart  Chapparo 
(in  preparation,  1989). 

Forty-eight  percent  of  the  1600  students  enrolled  in  the  Spring 
1989  Introductory  Psychology  course  opted  to  use  the  STAR  program. 
(It  is  noteworthy  that  an  introductory  psychology  course  shares 
important  characteristics  with  the  •'threat”  block.  There  is  a  large 
amount  of  information  contained  in  the  course,  and  the  students  are 
essentially  naive  about  the  material) .  In  assessing  the 
effectiveness  of  STAR,  students  were  ultimately  divided  into  those 
who  used  it  at  high,  medium,  or  low  levels,  or  not  at  all.  While 
STAR  offers  practice  quizzes  and  detailed  feedback  about  content 
areas  for  missed  questions,  it  is  not  now  being  used  as  the 
examination  of  record.  These  are  taken  in  an  associated  computer 
environment.  The  record  of  performance  on  these  "quizzes  that 
count"  is  presented  in  Figure  8.  As  is  readily  apparent,  the  more 
students  used  STAR,  the  greater  the  improvement  in  the  quiz 
performance  compared  to  that  of  the  non-users.  In  addition,  on  the 
final  examination  (required  of  all  students)  the  improvement  was 
even  greater,  16-,  17-,  and  26-%  improvement  for  low,  medium,  and 
high  STAR  users,  respectively,  compared  to  non-users.  Thus,  there 
may  also  be  substantial  long-term  benefits  as  well. 

Further,  the  course  as  a  whole  is  self-paced,  with  students 
taking  the  examinations  of  record  when  they  are  ready  for  them. 

Among  the  STAR  users,  those  who  finished  the  course  early  had  spent 
more  time  using  STAR  than  those  who  did  not.  Some  students  can 
actually  finish  a  semester  course  in  just  a  few  weeks.  Overall, 
those  who  used  STAR  performed  significantly  better  in  the  course 
than  those  who  did  not,  and  this  was  independent  of  the  students' 
academic  standing  or  grade  point  average.  So,  it  was  not  just  a 
matter  of  the  better  students  using  this  new  technology. 

Another  important  aspect  of  the  course  structure  is  that  of  unit 
mastery.  Students  are  permitted  to  take  chapter  quizzes  as  often  as 
they  wish  until  a  self-defined  level  of  mastery  is  achieved,  with 
their  grade  being  determined  by  the  overall  average  they  achieve. 
They  take  the  quizzes  when  they  are  ready  and  then  move  on.  It  was 
noted  in  the  89-11  class  survey  data  that  those  students  would  have 
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preferred  that  system  as  well.  Therefore,  the  concept  of  self- 
paced  unit  mastery  and  the  use  of  a  computerized  tutorial  such  as  an 
adaptation  of  STAR  might  be  incorporated  in  its  entirety  into  the 
"threat"  block  of  instruction.  That  would  offer  several  important 
advantages: 

1.  Students  would  use  STAR  to  guide  their  study  through  the 
course  material,  with  STAR  providing  feedback  and  remedial  advice  on 
an  individual  student  basis. 

2.  Students  would  be  able  to  take  the  quizzes  on  a  per  CAE 
basis,  and  when  they  were  prepared  for  them  (although  certain 
deadlines  would  have  to  be  set) . 

3.  STAR  would  provide  an  ongoing  instruction  and  review 
capability  during  the  evenings  and  weekends  so  students  would  not  be 
"in  limbo"  during  this  valuable  study  time  (assuming  PC  access 
during  these  times) . 

4.  Students  would  be  able  to  use  STAR  as  a  reviewing  tool  when 
they  proceed  to  other  blocks  of  OBC,  and  need  to  sharpen  their 
"threat"  skills. 

5.  STAR  does  not  need  complex  computer  equipment.  As  mentioned 
earlier,  it  will  run  on  any  IBM  compatible  machine.  For  what  may  be 
a  very  cost  effective  adaptation,  several  workstations  could  be  made 
available  in  a  relatively  small  space,  and  for  comparatively  little 
cost. 

It  is  ironic  that  the  multiple  testing  on  dropped  CAEs  that  now 
takes  place  is  itself  a  crude  analogue  of  the  unit  mastery  system. 
What  is  recommended  here  is  that  this  "naturally  selected"  process 
be  systematically  designed  and  incorporated  into  "threat" 
instruction.  A  very  important  point,  however,  is  that  while  the 
usefulness  of  a  tutor  such  as  STAR  may  seem  apparent,  it  should  not 
be  adopted  until  an  appropriate  study  has  been  conducted  to 
determine  its  effectiveness  for  an  information  dense,  two  week 
course  such  as  "threat." 

Finally,  while  "threat"  is  a  vital  part  of  the  OBC  curriculum, 
and  the  recommendations  presented  above  are  offered  to  improve  the 
acquisition  and  retention  of  information  in  that  block,  it  would 
seem  that  more  data  analysis  is  needed  on  the  other  blocks  of  the 
MIOBC  course  as  well. 
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